Training a Sigmoidal Node Is Hard

This paper proves that the task of computing near optimal weights for sigmoidal nodes under the L regression norm is NP Hard For the special case where the sigmoid is piecewise linear we prove a slightly stronger result namely that computing the optimal weights is NP Hard These results parallel that for the one node pattern recognition problem namely that determining the optimal weights for a threshold logic node is also intractable Our results have important consequences for constructive algorithms that build a regression model one node at a time It suggests that although such methods are in principle capable of producing e cient size representations e g see Barron Jones nding such representations may be computationally intractable These results holds only in the deterministic sense that is they does not exclude the possibility that such representations may be found e ciently with high probability In fact it motivates the use of heuristic and or randomized algorithms for this problem